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Estimation  in  the  Presence  of  Noise  of  e  Signal  Which 
is  Plat  Except  for  Jumps  •  Part  II «  The  Empirical 
Bayes  Approach 

Abstract 

This  is  the  second  of  a  two-part  paper.  In  the  first 
part  Yao  (1912) «  a  special  Bayesian  Model  A  is  studied  in 
detail.  In  this  part,  a  more  general  model  is  proposed 
and  studied  in  an  empirical  Bayes  framework.  The  results 
for  Model  A  are  applied  to  step-function  signals  usin? 
the  ideas  of  empirical  Bayes  and  maximum  likelihood 
applied  to  the  parameters  of  the  Bayesian  Model  A.  An 
efficient  computational  method  is  proposed  to  approximate 
the  likelihood  function  under  Model  A.  several  empirical 
Bayes  estimators  of  the  unknown  step-function  signal  are 
eomparsd  by  simulation. 

Key  words:  Change  points,  nonlinear  filtering,  smoothing, 
empirical  Bayes,  maximwi  likeliho^,  pseudo 
msxisnas  likelihood,  Kullbeck  Information. 
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1.  Introduction 

This  is  the  second  of  a  two-part  paper.  We  consider 

the  problem  of  estimating  a  signal  which  is  a  step  function 

when  one  observes  the  signal  plus  noise.  In  other  words. 

in  discrete  time  denote  the  signal  Process  by  Uj.u^ . u,j. 

and  let  except  for  occasional  changes.  Let 

the  obsegyatlons  +  c^,  1  1  where  the 

are  noise,  we  are  interested  in  estimating  u  based 

n 

on  1  1  i  <  T.  In  the  first  part  Yao  (1982),  we 

studied  this  problem  in  a  Bayesian  framework.  A  special 
Bayesian  model  (to  be  called  Model  A)  was  proposed  there 
and  the  corresponding  Bayes  solution  was  derived  and 
evaluated  analytically  and  numerically.  In  the  second 
part,  we  will  Invoke  the  idea  of  esplrical  Bayes  to  attack 
more  general  esses  where  not  all  of  the  sssunptions  of 
Model  A  are  sstisfisd. 

In  the  next  section,  a  gensrslisstion  of  Model  A  la 
proposed.  In  Section  3,  partial  results  are  obtained  on 
identifying  the  underlying  distributions  and  estimating 
optimally  the  step-function  signal.  In  Section  4,  the 
results  for  Modal  A  are  applied  to  more  general  step- 
function  signals  using  the  idees  of  empirical  Bayes  and 
iMxiiram  likelihood.  In  Section  5,  en  approximation  to 
the  likelihood  is  proposed  and  this  epproxlmatlon  is 
svslusted  in  terms  of  the  Kullbeck  infonsation.  In 


Section  eeveral  en(>irical  Bayee  eatlmatora  are  studied 
by  use  of  simulation. 

2.  A  General  Bayesian  Wodel 

In  this  section  «e  propose  the  following  model. 

(1)  The  time  intervals  between  successive  changes  in 
the  signal  are  l*i.d. 

(2)  The  sueoesslve  heights  of  the  signal  are  i.i.d. 
O)  The  additive  noise  is  an  I.i.d.  sequence. 

To  be  more  specific^ 

(IM  Let  C  ^  i'i.d.  (F^)«  positive 

Integer  valued  with  finite  first  moment  .  Let  t*  be 
Independent  of  and 

Pr(C*«i)  -  Pr(5>l)/*5  1-1,2,... 

Define  the  sequence  of  change  points  (n^)  by 

=  o,nj^  St' ,  5  wn,^5 


(2*)  Let  y,  be  i.i.d,  (F^)  and 

represent  successive  heights  of  the  signal,  i.e.,  define 
the  aiqnal  process  {\)^\  to  be 

-  \  V  "  4  \*l 

(3*)  Let  the  additive  noise  e  »t2, . . .  be 
i.i.d.  <Pg).  Msume  Be  *  0.  Let  the  observations  be 


Wote?  Model  A  la  a  special  case  of  this  general  model 
when  is  geometrical  and  Fy  and  F^  are  normal. 

Model  A  can  be  described  by  four  parameters  P,e,o,o^ 
>il»re  Pr(t-i)  -  pd-p)*"'  .  an,  r^,.*  (O.a^*) . 

Suppose  F^ ,  Fy ,  and  F^  are  not  known .  Two 
natural  queationa  arise: 

(01)  Are  they  identifiable? 

(02)  Can  be  estimated  "optimally*? 

n 


Mote*  The  random  variable  f*  is  Introduced  in  order  thst 
the  0*1  sequence  generated  by  (njfn^****^  (ones  et  the 
n^  snd  teros  elsewhere)  be  stetionsry.  This  is  s  matter 
of  eonvenienee  end  Is  not  essential  as  fsr  ss  asys^totle 
rssults  are  concerned. 


me  designate  the  aubaequence  {B^ri<,njcj)  by 
and  we  ahall  call  eatlmataa 

uniformly  asymptotically  i  relative  to  F^,Fy,  and 


;ial 


*  iil  *<F5.FY.%>’*('t.%.Ft)  '"nl*!'  '“n’^ 


The  proofs  of  (3.1)  and  (3.2)  appear  in  the  Appendix. 
We  have 

(3.3)  ElfjXj*  0^(Yj*tj)“  (Yj+tj)**(  l-0j)E(Yj«  Cj) '  (Y  j*c  j) ' 


unlfomly  for  n«l«2«...«T. 


Pj  •  Pr(f*  1)  and 


t^re  e,«  «,  •  ,  wans  expectation  aceordlne  to  the 

probability  structure  determined  by  and  P^. 

This  definition  is  consistent  in  essence  with  that  in 
Robbins  (1964). 

3.  Partial  Ansvers  to  (Ql)  and  (02) 

Proposition  3.1  (Strong  Consistency  of  p^) 

Assume  that  /x®dr^(x)<-,  /■x®dP^(x)  <•,  Var(Y)>0, 

and  <  *• .  Then  there  is  an  estimate  such  that 

-*P^  w.p.l. 

Proof  of  Proposition  3,1 

Although  <X^)  is  not  a  meahly  dependent  sequence* 
the  following  are  still  true. 


(3.1)  llK  i  i  X® 
T—  ^  nil  " 


1  <  «  <  4. 


(3.2)  lim 


»  nil*"’'"** 


ft  >1*  o  4  6  <  4. 


(3.4)  ?  Pr(^‘  >  I  *1) 


I 

i*k 


1  (l-l)p./Et 

i*i^i  * 


•  ( 


i 

i-i+ft 


^Pi 


ft  l  Pi)/EC 

l-l+ft  ^ 


ft  • 

-  (Bt  -  I  ip.  -  ft  I  Pi)/B5 
1-1  ^  i-l+ft 


•  |BC  -  1+ 


ft>l 

I  (t-Dp.l/Bt  , 
i*l  ' 


In  particular «  <  1. 

Substituting  in  (3.1)  and  (3.2). 

T  ♦ 


Jr^ 


I 
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(3.51  AjI  i  EY  .... 

(l.t)  A,  J  X*  ->  EY^  ♦  ec’  «... 

n*l 

(3.1)  A33  I  I  x’  ->Ey’  ♦  3EYEC*  ♦  Et’  .... 

n*! 

(3.*)  A.si  I  X*  ♦  EY*  ♦  .EY^Et’  ♦  AEYEc’  +  Ec‘  .... 

*  ^  n-1  " 

(3.9)  Bj.  J  V  Vn*l-  <’l«’  *  •  •• 

(3.10)  »y-  I  X*X„,3  *p,(BY^Ee*EY)  ♦  (1-Oj)  (EY^EY.Ee^EY)  .... 

n*l 

(3.11)  83=  jf  T  x’x„^i‘Pi(EY«.lE.*EY*.Et’EY) 

♦  (I-O3) (tY*gY+JEc*(EY)*+Ec*EY)..». 


T«1 

(3.12)  B^I^  I  X*X^^3  *  Pj(EY*+2Ec^EY^+(Ei:‘')^) 
n*l 

♦  (l-pjl ((EY^)^+2Et*EY**(Et^)^l  ».s. 

VBlng  (3.»)  througli  (3.12)  ««  .hall  .(low  that  pj 

can  l>e  con.i.tentlY  ctimatad.  Plr.t  it  may  be  seen  that 

n (EY*)  5 (EY*) *• 2 (Xj-Ej) * lEY*) 1 3Aj (Bj-aJ) tAj (2A3B3-2aJ*AjAj-BjI 
+  AjAj-Aj-BjtB^)  ♦  IAj)A3-3AjA^) (Bj-aJi-aJaj+aJbj 

-  Aj(B3-A3)-Bj(AjA3-Aj)  ♦  (Bj-B,)aJi*  0  (T*-)  a... 

The  cave  where  pj  «  0  it  vpeclel*  for  then  >  1  and 
e  change  takev  place  at  each  tine  point.  Then  there  is  no 
way  to  diatlngulvh  the  algnal  frost  the  nolve  without 
additional  Infonsatlon.  Por  this  reason  we  cmtalder  two 

cases: 

U)  hhen  eatinate  by  0j^=  0  and 

therefore  eatinate  P^  by  6^,  the  distribution  with 
unit  nasa  at  1. 


12)  Mhen  1  aatliuta  EY^  by  t)ie  larqer 

Bolutiofi  (denotad  by  BY^I  of  tha  quadratic  aquation 
(KEY*)  -  0.  Since  niA?)  •  0,  EY*  >  A*  It  will  be 


(3.»),  aatiaata  Pj  by  OjS  ■ln(  (Bj-Ajj/IEY^-A*)  ,i)  .  By 
(J.4),  eatiaiata  EC  by  ift  f  l/<l-o,).  By  (3.2)  and 

T-t  *  A 

(3.3),  aatinate  o,  by  o,  =  If  I 

for  2  <  f  1  Tl^l.  Applying  (3.4)  which  rolates 
0|  and  p|i  wc  ar«  led  to  Introduce  p^  recursively 


p,  5  2  -  E5(l-P.) 


*  *  »  ^  *  ...  A 

Pj,  S  I  ip.4(k+l)(l-  I  Pa)-*5(1-Pw,)  k  -  2,3,..., 
*  i-1  ^  i-1  ^  *  * 


Let  os  estiMte  p^t  p^f  •••  ^ 


a.  *  *  m  •  A 

p^  5  SMxfjslnf  I  Pi  ♦  Pfcil)  -  i  PjiO),  k-2,3, 
«  1.1  ‘  *  1-1  ^ 


^C1o9T|41  -*  jij 


flogTI^ 

aL  ^ 


P.  ?0,  k  >  IloqT)  4  1. 


There  are  two  different  cases: 


(i)  If  PrKj  -  1)  -  1,  i.e.  -  0,  then 


Aj*  ♦(BY)*  (T»-)  a. 8. 


Bj  aPjEY*  ♦  (l-Pj)(EY)*  -  lEYI*  (T— )  a. 


Since  (  X^)  is  i.l.d.,  we  can  apply  the  law  of  the 
iterated  logarithm  (Philipp  and  Stout  (1975),  p.  26), 

Aj^  -  Sj  -  Ot  /lo^ logT/T  )  a.s. 

Thus,  for  T  large  enough.  So 

-  6|  for  T  large  enough.  This  proves  the  consistency 

when  F^  ■  6^. 


0,  thMi 


(tl)  If  Pr(Ci  *  1»«  1.  i-«'  Pi  > 


lla  a!  •  (IT)*  <p,BY*  ♦  (l-OkXBt)*  •  11a  B.  a. 8. 

T.—  *  ‘  T  T.-  * 


Aj*  for 


oBlnq  tbe  aBaaavtlon  Var(T)>0.  Bo  Bj 


ii 
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E  (E^  (u^ 


for  n«k^l, , . . ,T-c(T) 


where  B 

n 

S  ,,n*c(T) 


•  cxpactatlon  under  c(TI», 

iC(T)),  c(T))  end 


e(T)). 


therefore,  we  need  only  show 


“;'<^n'“k*ll>'l’‘*‘>  -  -k+l'’  •  -  Xkrl'*- 

Obviously, 


^>l'‘'kell  *!'"■*>  **'''kell  ’'f**'  'f-'  «•*• 

Since  In  bounded  by  ■e*(|5|,|Xj|,....|Xj^^j|), 

1^  e(Vi‘«keil*f**>  *  »k*i>*  ■  ■<*'Xkei I -  "kn'* 
by  the  donineted  convergence  theerea.  a 


This  proof  of  Proposition  3e2  f^leost  estsblishes  the 
anlfora  esynptotlc  optisielity  of  ••  •" 

estlaste  of  for  k+l<n<T-ctT)  and  k  larpe. 


kemrlt  2t 

Although  in  Proposition  3.2  and  P^  are  required 

to  be  Gaussian,  the  same  result  can  be  proved  if  they 
belong  to  (regular)  parametric  families  of  distributions 
whose  parameters  can  be  estimated  consistently. 

4.  An  Empirical  Bayes  Estisate  Using  Model  A  with  Unknown 

Parameters 

In  general,  a  atep-funetion  aignal  can  be  either 
deterministic  or  stochastic  and  therefore  Model  A,  or  even 
the  general  model,  can  fail  to  be  aatisfied.  Why  then  should 
we  eoneider  these  models?  The  basic  idea  is  that  it  is 
hoped  the  unknown  signal  would  resemble  a  "typical*  real¬ 
isation  of  these  models  with  properly  assigned  parameters 
or  distributions.  Indeed,  this  ia  a  posaible  interpretation 
of  the  es^lrlcal  Bayes  idea.  The  moat  famous  example  ia 
ths  Jamaa-Btein  eattmata  which  showa  aoaie  superiority  to 
the  claaaleal  estimate  of  the  mean  of  a  multivariate 
normal  distribution  . 

It  ia  almpet  Impossible  to  produce  a  sensible  estimate 
of  the  signal  without  any  information  about  the  structure 
of  the  eignal  and/or  the  noise.  Bence,  our  first  assumption 
is  that  the  noise  is  Geoaetan  whita  noiaa.  One  sksin  reason 
to  have  tha  Gaussian  aaaw^ion  la  that  it  ia  hard  to 
diatinguiah  outliers  frcai  jumps  if  tha  noise  has  a  heavy 
tailed  distribution.  Puthensore,  if  the  step-function 


1 


( 
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siqnal  has  many  jui«pa«  the  nolee  variance  cannot  he  well 
estimated.  Indeed*  the  noise  variance  in  Hodel  A  Is 
not  identifiable  without  further  Information.  For  instance* 
the  observation  process  (X^)  is  i.i.d.  N  (0,1)  when 
•  (1, 0*1*0)  or  (p*0*0*l)  .  So.  we  isake  the 
second  assumption  that  the  rate  of  ^he  signal  is 

at  most  p^  where  is  a  specified  number  between  0 

and  1, 

As  the  next  step  in  generalising  our  estimation 
procedure*  let  us  assuias  that  Nodal  A  applies  with  un» 
known  parameters  p.O.o.o^  and  apply  maximum  likelihood 
to  estimate  these  parameters.  To  be  more  precise,  we 
estimate  the  signal  as  follows.  First*  fit  Nodal  A 

to  the  observations  X|^(1<1<T)  by  finding  the  maximum 
likelihood  estimates  (NLE)  p*9*o  and  with  the 
constraint  that  p  <  Pq*  Next*  estismite  by 


o(aXj*c,...  -  |a|o  (Xj . x^) 

Oj(aXj*c . aX^.c)  .  |a|  (Xj, . . .  ,X^) 

and  since  Model  A  is  time  reversible,  we  have 
Proposition  4.1 

The  empirical  Payea  estimator  of  u^,  is 

translation  invariant,  scale  Invariant  and  time  reversible. 
That  is* 

I“(aXj*c,...,aXT*c)  -  a  u“lXj, . . .  .X^)  ♦  c 
«”(Xj,...,X,)  -  (Xj,X^.j,...,Xj). 


(4.1)  .  e 


Where  BB  atmda  for  aavirieaX  Bayas. 

Bince  the  NLf  aatlafy,  (for  oonatanta  a  f  0*e)* 

P(aXj+c . ax^+c)  -  p(X^*...*X,) 

14.2)  e(aX2ee,..,*a 


The  computation  of  the  llz,E  can  be  very  time' 

T 

ctmsuning.  A  naive  method  may  require  0(2  )  operations 
to  compute  the  likelihood  for  each  quadruple  (p,0,o,o^). 

We  present  In  Proposition  4.2  a  representation  of  the 
likelihood  function  which  reduces  the  nunber  of  operations 
to  the  order  of  T^.  Since  the  leg  likelihood  L(p,e,o*o^X^) 
setisfies 


iX^+c)  •  aO(Xj»....x^)  ♦  e 


(4.3)  L(P,8,o,a^,xJ)  -  L(p,0.o/ffj.,l|  (XmJ)  -  T  log 

where  X*^  •  eeed  only  coneider  L(p,0.o.l;xj> . 

let  Sj=  0  end  S„  3  J  x,^  for  1  <  "  i  T- 
Proposition  4.2 

I*(p,0.oa;xT  -  *T)  -  log  f  (Xe)  +  log 

11  Xj  1  Xj^^j  n*l  1  1 

i«horo 

KXj)  •  X(0,o*+1), 

(4.4)  i(x,.,ix")  •  (1-p)  [  A*"’,  x-*)— ar.»5.  ,  __L_  ♦  n 

"  *  ‘  X-l  ^  k*a  *  kto^ 

♦  p  »(«,a*  4  1) 

end  »*jj*  ere  defined  in  Propoeition  4.X  of  Teo  (HM) . 

Pr<yof  of  Proposition  4. a 

Us  n««d  only  4oriv«  (4.4).  flovoxvrr  this  is  s  simple 
conssqmsncs  of  Proposition  4.2  of  Too  (19421  snd  the 
following  identity. 

l(X„^l  Ix^)  -  4  .„^ll  xj) 

•  in-p)i(«„|xj)  ♦  p  »(o,9*)i  •1(0,1) 


-  (l-p)l(v^lx^)®»((0.1)  ♦  p  M(0,c’-H) 

where  o  ^  the  convolution  of  law  with  1-2 • 

S.  An  Approximation  to  the  Likelihood  and  the  Pseudo  MLE 
Even  though  Proposition  4.2  suggests  a  way  to  compute 
the  likelihood  with  O(T^)  operstions.  it  is  still  time- 
consuming  to  compute  the  MLE  without  further  reduction  in 
computstion.  Therefore  it  is  desired  to  find  a  store 
efficient  way  to  approxlisate  the  likelihood.  Me  will  make 
use  of  sn  ides  of  Harrison  snd  Stevens  (1976)  to  develop 
an  approximation  procedure  which  reduces  the  number  of 
operations  to  the  order  of  T.  This  idea  has  been  used  snd 
justified  in  Yao  (19B2). 

NS  approximate  UX^^2  I  **  follows,  hgsin*  assume 

0  and  ■  1  for  aimpliclty.  tn  Section  5  of  Yao 
(19B2)p  N  (^>T  la  introduced  to  approxisuite  i 
where  6^  and  are  defined  recuraively.  Since 

I  -  tl-p)»*(u„ix")0h  (0,1)  +  pM(0,o^+l) 

we  are  naturally  led  to  approximate  ^^1^ 

Cl-P)  iKOnxTj+l)  ♦  pWO.o^+l). 

Mow  «s  can  approximate  tWa  log  likelihood  L(p.  O.o.o^tX^) 
by  use  of  Propoeition  4.2  and  the  above  epproxlmatlon  and 
denota  this  approximate  log  likelihood  by  Zip,9  ,o  , 

It  should  be  noted  that  this  epproximstlon  is  exact 
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when  p  is  0  or  1»  for  Mod«l  A  is  s  Gaussian 
system  when  p  is  0  or  1.  Now  we  propose  to  measure 

this  approxiskation  in  terms  of  the  Kullback  information 
between  exp(L)  and  exp(L)  under  Model  A.  More 
precisely*  we  will  treat 

(5.1)  l^(p,e,0,0j) 

as  a  measure  of  how  well  L  is  approximated  by  t.  Note 
that 

(5.2)  In(p,9,o,Cj.)  ■  In(p.O,a/a^,l) 

Me  considered  63  cases  where 

pe(0.02,  0.05*0.1*  0.3*  0.4*0. 6*  0.8)  *ac(0.S*  1*2,3*4.5*7*10*15}  * 
6  ■  0  and  1.  The  were  eotimated  by  use  of  sim¬ 

ulation  with  a  computer  (HP  3000)  where  400  samples  of 
else  T  •  30  were  generated  for  each  case.  The  results 
are  presented  in  Table  5.1. 

According  to  Table  5.1*  B(L-L)<  0.14*  and  80(L-U<  0.48. 

bsre  80(Y)  is  the  standard  deviation  of  random  variable 
y.  8o* 


-1.44<  E(L-&)  -  3  8D<L-I.>  <E(Zr-£)  *  3  8D(L-L)«  1.58 


liie  probability  thet  the  likelihood  ratio  enpCL-L)  satisfies 
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0.24  ^  exp(-1.44)  <  exp(L-t)<  exp(l.S9)*  4.85 
is  very  high  in  the  worst  case  under  Model  A.  This  suggests 
that  the  approximation  will  yield  reasonably  good  results. 

Me  shall  define  the  pseudo  MLE  p',S'o*^^  as  the 
values  of  the  parameters  which  maximise  £  subject  to 
p  ^  p^.  Then  we  estimate  by 


'*•’>  =n  =  '(p-.s'.ais;  )  '“n'*!'- 

^ _ Simulation  on  Empirical  bayea  kstimatora 

Xn  the  last  section*  we  have  Introduced,  for  the  sake 

«» ♦  “EB 

of  computation*  whi^  is  sn  approximation  to  . 

Xn  order  to  evaluate  the  performance  of  we  carried 

out  the  following  computer  simulations  on  an  HP  3000. 

Ms  oonsidsrsd  31  dstsrminlatic  signal  sequences 

of  length  T  •  20  (l<n<20*  l<i<21).  For  each 
n  —  —  — 

signal  ssqusncs*  ws  generated  100  samples  of  Gaussian 
whits  noise  of  vsrisnce  1. 

In  defining  u^,  we  estimated  the  psrsmetere  of 
Model  A  by  use  of  pseudo  amutimum  likelihood.  It  is 
interesting  to  see  how  well  the  method  of  moisents  can  do 
compared  to  the  pseudo  fMximum  likelihood  method.  It  is 
also  Interesting  to  see  hoc  much  the  additional  Information 
0  *1  can  contribute  to  sstimstlng  u.* 
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Hence,  we  considered  the  following  four  estimators  of  u 


Pj  *  max(min(p^f0.2)  >0) ,  6^  ■  X,  Oj  »  max( 

and  p^,  04  satisfy 


(i)  Estimator  1  -  u  .  o  «  0  7 

(11)  Estimator  J  -  This  Is  dsflned  In  ths  same  way 
as  Eatisiator  1  axcept  with  one  more  constraint  •  1 
In  the  pseudo  maximum  llhellhood  estimation  of  the 
parameters. 


X^/T  .  *  sj  ♦  1 

T-1 

I,  Vn.l'''’-‘>  ■  *  'l-Pt’-’l 

n*i 


(ill)  Estimator  3  *  E/ 

Pj  -  max(mln(p2,0.2)  ,0) ,  dj  •  Jf 
Oj  -  max(  Oj,0),  0^^  >  max (  o^j, 
satisfy 


'‘’cl’ *“"*’'*’ 

(the  sample  mean!, 

0)  and  P2*°2’^t2 


1 


n“l 


*0 


2 

c2 


A.  *n  ''n4l/'»-l)  ■  ♦  (1- 


Jl  *nV2/'T-»'  '  ♦  (l-Pj)*o,*. 

(Iv)  Eittmator  4  -  E 


(Pj-ej.Oj.o^j) ''‘nl’'l> 


We  use  the  average  of  mean  squared  errors  (KHSE)  as 
the  criterion.  The  simulation  results  are  presented  in 
Table  6.1  where  we  also  present  the  mean  and  standard 
deviation  of  «  the  pseudo  Wl*E  of  o^.. 

Woft 

Ml  the  four  estimators  have  one  coewon  property. 

That  is  , they  first  estimate  p,0sO,Og  and  then  estimate 

T 

Ujj  by  ths  corresponding  Bayes  estimate 

In  the  simulation  above,  we  actually  coiqputed  the  approximate 
Bayes  estimate  (see  Yao  (19B2),  Section  S)  Instead  of  the 
exact  one. 

nemerkss  (Based  on  Table  6.1) 

(1)  Boughly  speaking,  when  the  number  of  jvn^B 
Increases, the  MSB  of  Lnereeeee.  When  the  sire  of 

juar*  increases ,  the  MSB  of  u^  first  Increases  and  then 
decreases.  For  when  the  site  of  jimp*  is  moderate  (l.e. 


% 


co«p«ti«bl«  with  the  nolee)  it  it  hard  to  tell  where  jumps 
take  place  and  to  take  appropriate  action.  This  property 
it  similar  to  that  of  the  Bayes  estimator.  (See  kemark 
1  of  Section  7  in  Tao  (1912)1. 

(2)  Estimator  1  (u^  )  It  better  than  Estiinttor  3. 

This  implies  that  the  method  of  pseudo  maximum  likelihood 
it  significantly  better  than  the  method  of  moments  in 
finding  suitable  parameter  values. 

(3)  Estimator  1  is  just  slightly  worse  than  Estimator 
2.  So  the  information  about  the  noise  variance  is  not 

very  important  for  estimating  the  signal  unless  the  rate 
of  change  in  the  signal  is  high,  in  that  case,  it  is 
hard  to  estimate  well. 

(4)  The  empirical  Bayes  estimator,  is  robust 

against  the  signals'  behavior.  However,  it  is  not  known 
how  to  deal  with  cases  involving  non-Cauasian  noise  which 
may  introduce  outliers  under  the  veil  of  jwps. 

(5)  When  the  prior  information,  the  rate  of  change 

<  pQ,  is  not  correct,  may  be  misleading,  although 

our  limited  simulations  do  not  indicate  so. 

(S)  It  is  Interesting  that  ,  the  pseudo  NLE 
of  estiisates  well  with  small  bias.  This  is 

essentially  due  to  the  information  P  <  Pq* 
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E( 


N  N—  1 

I  V  -  N  EV,^  *2  r  (M-k)  kV  V 
n-1  "  ^  kil 


,  T- 1  T- 1. 

tlm  i  I  I  .  F  V„l  -  0  a. 

T**  n»l  n») 


N-l 


EV/  ♦  2  I  (H-k)  eV.V.  ♦  2  I 
*  k-1  ■  ^  *  *  k*» 


1 

'  Jy 

•f*m  n»J 


tin  s  lEY*  c  ♦ 


,  I  N-l 

<  N(ev/+2  I  |EV,V.  1+2  T  Pr(no  k,  •ucb  that 
1  k«l  ^  *  *  k»l+l  ^ 


d-1 

r  V  »nax(n. *  max(T- 
k*l  K  1 


l+lin,j^<l+k)E(YiY2-(EY)^I  (YjYj-(EY)^H 

t  N-l 

-  HIEV  *  ♦  2  I  l*V.  V*,|*2  r 

^  k-1  ‘  *  k-t+l 

Pr(t'>k-l) (EY)*V«r<Y)l 

I  '  2 

«  HIEVj*  +  2  J  |*Vj\jj^j|*2  ECMCT)  'fcr(T)) 


d  5  inf  fk  :  '  T) 


Now,  by  the  atrong  law  of  large  numbers. 


tin  X  •  E  E;  a.s. 


tin  I  •  ftM*U-lrO) 

T—  ° 


I  k  Pr(?;  -  k*l) 
k-1 


Ifimftont  by  the  '3a*l-RoknM  strong  law  of  large  numbers 


Therefore, 


fointa 
yf  Qtfnge 

r 

Bst.  1 

Btt.2 

fist.  3 

10 

10 

.254(.018) 

.241(.017) 

.670  (.058) 

10 

.187(.019) 

.185(.019) 

.4861.050) 

15 

.2Q4(.008) 

.197(.00l) 

.3141.027) 

15 

.301(.024t 

.273(.019) 

.7981.054) 

4.10 

.  370(.017) 

.361  (.016) 

.8021.054) 

7,14 

.407(.030> 

.3711,025) 

.9471.071) 

5,15 

,395{.n31) 

.3801.027) 

.9521.078) 

5.15 

.356(.022) 

.3501.022) 

.7931.070) 

.3tS(.033) 


.1I3(.018) 


.197(.007) 


.38S(.03€) 


.610(.040> 


.642(,D9D) 


.7I0(.0(T) 


.440(.03I) 


.930  .180 


.970  .165 


.961  .162 


.944  .204 


.936  .220 


.916  I  .258 


.9X3  .222 


.944  .198 


*1  Th«  nuinb«r  in  par«nth«i««  n«xt  to  an  ontry  1«  tho  ootlMtod  itaAdard  arror  for  that  antry. 

This  coluam  is  tha  AHSC  of  tha  aatisiator  usin9  tha  avara^as  of  tha  data  points  batwaan 
■uecasaiva  tina  points  of  ehan^a. 

•  *  * 
t  Tha  astisMtad  standard  arror  of  tfia  aati«atad  8(o^*)  la  8D(o^M/10. 

Table  6.1  -  Continued 


.944  .189 


.997  .204 


.908  .185 


1.014  .180 


.314 


1.014  .228 


.961  .200 


1.024  .222 


1.339  .268 


1.238  I  .280 


8,13 

.3051.020) 

.3031.018) 

.6001.043) 

.3751.024) 

4,10,16 

.3481.023) 

.3081.017) 

,7291,041) 

.4931.032) 

4,10,16 

.4771.027) 

.4661. 02$) 

1  .7661.051) 

.5041.031) 

4,10,16 

.3281  032) 

.3141.031) 

.7281.046) 

.3051.032) 

4,10,16 

.2801.007) 

.2651.000) 

.3651.026) 

.2591.008) 

4,10,16 

.S3S{.038) 

.434(.0M) 

.9641.052) 

.6931.058) 

3,7, U, 16 

.4261.018) 

.41»C.ei3) 

.9071.081) 

.5781.039) 

3,7,12,16 

,3781. 022) 

.M«(.023) 

.9841.034) 

.3641.022) 

.ty3(.033) 

1.3111.065) 

2.1241.033) 

1  .8541.024) 

.•3*<.033) 

1.0371.045) 

1.9201.035) 

:  8i9nal  19  is  tha  following,  •  0.9<n-'l)»  1  ^  n  ^  20 

t  Signal  20  is  tha  following,  •  n-1,  1  i  »  <  11*  •  21  -  n,  12  <  n  <  20 

*  8ignsl  21  is  tha  following,  u.  •  lO-O.l  <n-ll)*,  X  <  n  <  20 
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